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17. METHODOLOGY DEVELOPMENT FOR EVALUATION OF 
SELECTIVE-FIDELITY ROTORCRAFT SIMULATION* 


MAJOR WILLIAM D. LEWIS, D. P. SCHRAGE, J. V. R. PRASAD, AND MAJOR DANIEL WOLFE 


The value of rotorcraft simulators in providing 
increased safety, reduced operating/training cost, and 
enhanced mission training has been well documented in 
the past 20 years. Because of the increased emphasis on 
rotorcraft simulation, the FAA has launched a program to 
establish certification standards for rotorcraft simulators. 
This program is aimed at updating both rotorcraft simula- 
tor standards and the methods of simulator validation 
through objective and subjective tests. No methodological 
and acceptance criteria currently exist for the performance 
and handling-qualities assessment of rotorcraft simulators. 
In order to establish certification criteria, a planned 
research effort to quantify the system capabilities of 
“selective fidelity” simulators is required. This paper 
addresses the initial step toward that goal: the establish- 
ment of a method for defining the performance and 
handling-qualities acceptance criteria for selective- 
fidelity, real-time rotorcraft simulators. Within this 
framework, the simulator is then classified based on the 
required task. The simulator is evaluated by separating the 
various subsystems (visual, motion, etc.) and applying 
corresponding fidelity constants based on the specific 
task. This method not only provides an assessment tech- 
nique, but also provides a technique for determining the 
required levels of subsystem fidelity for a specific task. 
This provides a helpful tool for use in eliminating system 
suboptimization. 

In developing a method, our task becomes twofold: 
define rotorcraft simulators in terms of fidelity and then 
apply data-collection techniques to evaluate performance 
and handling qualities. With respect to fidelity, the current 
thrust of minimizing training costs focuses attention on 
the question. What is the required level of fidelity? As a 
general rule, procurement of new simulation devices or 
the updating of existing models consisted of fulfilling a 
wish list. If a state-of-the-art system was desired, state-of- 
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the-art subsystems were procured and integrated. It would 
not be inconceivable to have a high-fidelity visual and 
motion system coupled with a somewhat simplistic math- 
ematical model. After investing millions of dollars in the 
system, the pilot comments were still unfavorable, for 
example, “A very nice procedural trainer, but it just 
doesn’t fly like the aircraft.” In this case the system inte- 
grator has suboptimized the system. Unfortunately, there 
is no quantitative method for defining a required level of 
fidelity for a given simulation task. A method for assess- 
ing selective-fidelity simulators would provide the sys- 
tems integrator with acceptance criteria and would aid in 
preventing system suboptimization by defining required 
subsystem fidelity for a specific task. This paper proposes 
to approach this problem by defining a task-specific simu- 
lator classification system based on fidelity. With respect 
to applying data-collection techniques for evaluating han- 
dling qualities, ADS33, the emerging standard in heli- 
copter handling qualities, coupled with the U.S. Army 
Light Helicopter (LH) Demonstration/Validation Phase 
test results are used to define the following: 

1. Quantitative evaluation criteria. In general, data 
collection focuses on quantifiable items such as band- 
width, minimum and peak rates, and damping ratios that 
are useful in defining acceptable tolerances between 
actual flight data and simulation data. 

2. Qualitative evaluation criteria. In general, a rat- 
ing scale system for a specified set of tasks is outlined for 
pilot acceptance of the simulation. 

As depicted in figure 1, the fidelity requirement for 
any simulation device is inherently dependent on the 
given simulation task. 

The requirements for simulators in the civil and 
military fields have expanded greatly throughout the past 
decade. Along with that growth, the variety of simulation 
tasks has also increased. Tasks can be categorized as 
follows: 
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Figure 1. Fidelity dependence on type task. 


1 . Non-real-time research analysis 

2. Part-task simulation 

3. Part-mission 

4. Full-mission 

5. Interactive mission scenario (networked, 
multiple nodes) 

Encompassing these tasks, simulation devices can be 
broadly categorized into three types: research, training, 
and procedural trainers. Within these broad simulation 
types, the levels of fidelity for a given type of device can 
vary greatly. For example, using cockpit crew coordina- 
tion as our simulation task, a work station can be defined 
as a relatively low-fidelity research simulator. Yet, 
another simulator of the same type, such as the Crew 
Station Research and Development Facility (CSRDF) 
located at Ames Research Center, certainly has a higher 
level of fidelity for the same task. Thus, for a specified 
task, the user must be able to determine fidelity require- 
ments. Failure to properly determine these requirements 
can result in (1) unsatisfactory results owing to a lack of 


fidelity, and (2) satisfactory results but at a premium cost 
(suboptimization). 

Consequently, it is desirable to classify a simulation 
device in terms of its fidelity. This allows a user with 
defined, task-specific fidelity requirements to select a 
simulator of appropriate fidelity and eliminate the above 
problems. The Federal Aviation Administration (FAA), 
for example, qualifies airplane training simulators in 
terms of objective fidelity. Simulator classification by 
fidelity sets a basis from which the user community can 
identify the specific simulation device that optimizes their 
needs. 

The current FAA approach to the qualification of air- 
plane simulators is embodied in FAA ACM20-40B. A 
similar approach is being planned by the FAA for qualifi- 
cation of rotorcraft simulators. The FAA approach desig- 
nates simulators in four categories, levels A through D, 
based on increasing levels of objective fidelity. Simulator 
standards, objective validation tests, and functional and 
subjective tests are then defined for each category. For 
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airplanes, the standards, validation tests, and functional 
and subjective tests have been fairly well accepted by 
industry through a series of workshops. Rotorcraft simula- 
tors do not have such well-defined standards owing to the 
unique capabilities and complexities of the air vehicle and 
existing simulation technology. Development of the 
rotorcraft criteria will require extensive research and 
development. 

Unlike the FAA approach to simulator classification, 
this method quantitatively classifies a given type of simu- 
lation device in terms of objective fidelity and a 
simulation-task-dependent weighting vector (TDWV). 
Each TDWV consists of a weighting parameter per 
fidelity characteristic, that is, 


SIMRATINGtask(i) = [FIDELITY CONSTANTS] 
*[TDWV] 


where 


[FIDELITY CONSTANTS] = [C C ockpit C au dio 

^motion • • • ^visual! 


[TDWV] = [K^^pit Kaudio ^motion * ■ visual! 


T 


For example, an air-to-air combat task requires a sig- 
nificant weighting parameter for the visual characteristic, 
whereas, the instrument training task would not require as 
large a weighting parameter for the visual characteristic. 


Clearly, in general terms, the weighting vector will always 
be dependent on the simulation task to be performed. The 
fidelity of the simulation device is assessed by rating each 
component of the system. For the purposes of this 
method, a simulation device is described in terms of 
10 subsystems, with each subsystem having varying 
degrees of sophistication. 

In surveying current simulation designs and existing 
technologies, there are generally 10 subsystems which 
adequately describe a given simulation device: 

Cockpit Audio 

Motion Control system 

Mathematical model Environment 
Ground handling Mission equipment 

System latency Visual 

In each subsystem, it is possible to associate a level 
of fidelity with the degree of equipment/software sophis- 
tication. For example, a motion system that employs six 
degrees of freedom can be associated with high fidelity, 
whereas a fixed-base system can be associated with low 
fidelity. This association between fidelity and the subsys- 
tems defines fidelity characteristics. Subsequently, listed 
below are the fidelity characteristics (rank order; low to 
high) of the simulator subsystems that span the spectrum 
of fidelity. The fidelity characteristics are assigned 
respective values from 1 to 4. 


1 . Cockpit/crew station 

Simulated instruments 
Basic, generic-type instruments 
Partially simulated cockpit 
Full-up crew station 
3. Motion 
None 

2DOF (pitch and roll) 

3DOF (pitch, roll, and yaw) 
6DOF 

5. Mathematical model 
3 DOF 
6 DOF 

6 DOF w/simple rotor 
6 DOF w/complex rotor 
7. Ground handling 
No gear 
Rigid gear 

Simplified gear model 
Comprehensive 


2. Audio 
None 

Significant cockpit sounds 
Incidental sounds (precip., etc.) 
Realistic 

4. Control system 
No force feel 

Constant force (spring/damper) 
Partial duplication of actual force 
Complete duplication 
6. Environmental 
Clean air 
Discrete gusts 

First-order filtered turbulence 
Rotationally sampled turbulence 
8. Mission equipment 
None 

Communication only 
Communication/navigation only 
Complete 
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System latency 

10. Visual 



Non-real-time (offline) 

None 



Significant delay 

Field of view 

Dvnamic ranse 

Detail 

Minimal delays 

Workstation 

Day 

Low 

Real time 

75° horiz./30° vert. 

Dusk 

Medium 


90° horiz./40° vert. 

Haze/fog 

High 


Wider 

Night 

Very high 


Assigning a value to each fidelity characteristic of the 
simulation device allows us to quantify fidelity by form- 
ing the fidelity constants matrix. For example, the U.S. 
Army 2B38 UH-60 simulator has the following 
characteristics: 

1. Cockpit: full up crew station 

2. Audio: incidental sounds 

3. Motion: 6DOF 

4. Control system: complete duplication 

5. Math model: 6DOF w/simple rotor 

6. Environment: discrete gusts 

7. Ground handling: simple gear model 

8. Mission equipment: complete 

9. System latency: real time 

10. Visual: 90° horiz./40° vert, full dynamic range 
medium detail 

With the above characteristics, the UH60 training 
simulator’s fidelity constants matrix is 

[FIDCONST] = [Cockpit, C auc j, C mo t, Cf ee j, C m athi 

Cenv* Cgmd» Cmep> Glat, Cvis] 

= [4 3 443 2 3 44 3.25] 

For a given simulation task, minimum acceptable 
fidelity characteristics must be established in order to 
constrain the number of simulation devices eligible to per- 
form the task. For example, to conduct aircrew contact 
training, some form of visual system is a minimum 
requirement for the visual fidelity characteristic. Without 
a visual system, the device would be unable to adequately 
provide task training. Consequently, a FIDCONST m m 
matrix: 

[FIDCONSTmin] = {min[Ccockpit G a udio 
Cmotion • • • Gyisuall } 

is utilized to establish the minimum acceptable fidelity 
characteristics for a given task. Exemplifying this concept, 
the U.S. Army 2B24 instrument training simulator, 
although it has many high-fidelity characteristics, such as 
a 6DOF motion system, full-up cockpit, and a complete 


mission package, is not eligible for consideration as a 
simulator for contact training because it lacks a visual 
system 

The function 

SIMRATING t ask(i) = [FIDCONST] * [TDWV task(i) ] 
constrained by 

[FIDCONSTmin! = {min[C cock pn C-audio 

^motion . . . C v jsual]} 

permits classification of a type-simulation device with 
respect to fidelity. Given a simulation task, a 
FIDCONSTmin matrix and a TDWV are determined, 
either subjectively or through extensive research. Once 
the weighting vector is known,. a minimum and m axim um 
SIMRATING task (i) is calculated. Given this range of val- 
ues, the simulation devices can be classified in terms of 
fidelity for a specified task. The range of values is parti- 
tioned into five subranges, the lowest corresponding to 
poor fidelity and the highest corresponding to high 
fidelity. 

As an example, suppose the given task js instrument 
training and the hypothetical FIDCONST m j n and TDWV 
have been determined to be 

[FIDCONST m in] =[4 244321341] 

[TDWV] = [1 0.5 1 1 0.75 0.5 0,25 0.75 1 0.25] 

Multiplying [FIDCONST m i n ]*[TDWV] we find the min- 
imum SIMRATINGtask(i) to be 23. For the maximum 
SIMRATING task (i), we must multiply 

[FIDCONST max ] * [TDWV] 

where the maximum fidelity constant matrix 
[FIDCONSTmax! ls defined as 

[FIDCONSTmax! = K4... 4] 


1 


I 

I 

i 
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Thus, the maximum SIMRATINGtas^) is calculated to 
be 112. Partitioning this range of values, we can now form 
a task specific (instrument training) classification for sim- 
ulation devices based on fidelity. For this example: 


Fidelity 

Classification 

SIMRATING 

Excellent 

A 

94-112 

High 

B 

76-93 

Medium 

C 

00 

1 

c /1 

Low 

D 

41-57 

Lowest 

E 

23-40 


Within the scope of this method, the fidelity charac- 
teristics were limited to a range of 1 to 4 in order to pro- 
vide an equivalent weighting between characteristics. This 
general approach obviously cannot handle specifics of any 
single characteristic. An alternative approach to provide 
equivalent weighting between characteristics is to employ 
normalized matrices for each characteristic. This approach 
would allow a greater degree of flexibility in assessing 
each characteristic. For example, while assessing the 
visual system the user could include the use of texture, 
infinity collimation, display types, etc. over a wider range 
of values. This enables the visual system characteristic to 
be well defined in terms of its specific attributes. 

The approach has assumed fidelity constants. This 
implies that no coupling exists between the various 
fidelity characteristics. Anyone who has flown in a simu- 
lator with a high-fidelity visual system employing infinity 
collimation knows this to be untrue. With a fixed-base 
motion system, the aforementioned visual system will 
cause a perceived motion. The strength of the perceived 
motion will vary, depending on the fidelity of the visual 
system. This example would indicate some degree of cou- 
pling between the visual and motion characteristics. This 
interdependance may be better represented by use of a 
matrix. The terms of the matrix could be constants or 
variables. The exact form of the coupling would need to 
be determined through research. 

The method at this point allows categorization of 
rotorcraft simulators in terms of fidelity for a specific 
task, but leaves unanswered the means of evaluating the 
performance and handling qualities of the rotorcraft 
simulator. 

A simulator must be assessed in the areas critical to 
the accomplishment of the assigned mission task. These 
areas typically include longitudinal- and lateral- 
directional responses, performance in takeoff, climb, 
cruise, descent, etc. Objective tests are used to quantita- 


tively compare simulator and aircraft data to ensure that 
they agree within some specified tolerance. ADS33 speci- 
fies an absolute standard for actual rotorcraft stability 
behavior. Requirements for handling-qualities standards 
are quantitatively specified, often in terms of frequency 
responses. Subsequently, characteristics of frequency 
response, such as bandwidth, damping ratios, overshoot, 
and time-to-peak become the tools of quantitative evalua- 
tion criteria. The method of ADS33 is applicable to simu- 
lation as well, except now these quantitative tools define 
tolerances between flight-test data and simulation data. 

Historically, simulator performance has been evalu- 
ated in terms of the simulator’s original design specifica- 
tion. This specification normally requires the simulator 
designer to meet the aircraft’s flight-test data within 
specified tolerances. Paralleling the FAA’s approach, per- 
formance testing will include the following flight regimes: 
hover, vertical and forward flight climb, level flight, and 
autorotational descent. The method of performance testing 
will consist of classic test techniques as outlined in 
USNTPS-FTM- 1 06, Rotary Wing Performance, refer- 
ence x. Tolerances between actual and simulated flight 
data are then established for each phase of flight based on 
simulator category. The tolerance for a category A simu- 
lator is thus the most restrictive and the tolerance for a 
category E simulator is the most relaxed. Figure 2 illus- 
trates the relation between the level of tolerance and the 
simulator category. The level of tolerance, represented by 
the expanding circles, reflects an increasing ± tolerance 
range with decreasing simulator fidelity classification. 
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Although classic performance testing techniques were 
adequate for evaluation purposes, classic handling-quali- 
ties testing techniques do not provide adequate informa- 
tion for assessing comparative simulator response. For the 
past 8 years, the U.S. Army, with participation from the 
other military services, the FAA, and industry, has been 
developing a new approach to specifying flight-handling 
qualities for rotorcraft. The existing military specification, 
MIL-H-8501 A, was first published in the early 1950s and 
was revised once in the early 1960s. The new specifica- 
tion will eventually be designated MIL-H-8501B; how- 
ever, for application to the U.S. Army LH procurement, 
the designation ADS33 has been issued. The approach in 
this new specification is based on defining mission task 
elements (MTE’s) and relating the visual cue environment 
(VCE) experienced in the aircraft to the level of stabiliza- 
tion required. Although the approach is currently being 
applied to qualifying rotorcraft, it will have substantial 
applicability to rotorcraft simulators. ADS33 provides 
clear quantitative requirements for classifying rotorcraft in 
terms of their handling qualities. A designation of 
levels (I, II, III) is utilized. These requirements are 
divided into three main categories; control-system charac- 
teristics, hover and low speed, and forward flight. Apply- 
ing this same standard to simulation, these categories now 
define evaluation criteria for simulation devices. Subse- 
quently, a set of tolerance levels between flight and simu- 
lation data must be established for each simulator cate- 
gory as described in figure 2. A set of flight-test maneu- 
vers based on mission-task elements is simulated to obtain 
quantitative and qualitative data. These quantitative data 
are then analyzed, and a comparison with actual flight-test 
data is conducted. The deviation between actual and simu- 
lated flight data then becomes the measure of acceptabil 


ity. The proximity to the specified tolerance then validates 
the simulation device classification. 

Pilot acceptance is a subjective evaluation. Subjective 
tests are designed to provide a basis for evaluating simu- 
lator capability to perform over a typical training period 
and to verify correct operation of the simulator instru- 
ments and systems. With respect to ADS33, the flight 
maneuvers outlined in the previous paragraph serve as the 
vehicle for a subjective, qualitative evaluation. Based on 
mission-task elements and the visual-cue environment, 
this set of flight maneuvers allows the pilot to assess the 
perceived performance and handling-quality characteris- 
tics of the simulator. These are then compared with the 
pilot’s assessment of identical maneuvers in the aircraft. 
This set of flight maneuvers allows the pilot to explore the 
perceptual fidelity of the system so that a fair assessment 
can be made. A Cooper-Harper rating scale system is used 
for the evaluation. 

Conclusion 

The method discussed here offers the rotorcraft simu- 
lation community a unique tool for analyzing and tailoring 
simulation devices for specific requirements. By tying 
fidelity directly to the simulation task, linkage is achieved 
through the simulator classification model. Concurrently, 
methods for evaluating quantitatively and qualitatively the 
performance and handling qualities of a rotorcraft simula- 
tion device are presented. These methods are consistent 
with current evaluation criteria. Additionally, this 
approach permits melding of the FAA certification 
method with the emerging rotorcraft handling-qualities 
specification, ADS-33, 
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PANEL DISCUSSION SUMMARIES 


NASA/FAA HELICOPTER SIMULATOR WORKSHOP 






FOREWORD 


These summaries were developed from transcriptions of stenographic recordings of the session presentations and 
discussions. Although care has been taken to identify and eliminate the errors of interpretation to which this technique is 
prone, undoubtedly others remain. They are the responsibility of the editor; the discussion moderators and panelists are 
blameless, at least in this regard. 


1 . Session A: Training: Limits, Allowances, and Future 

Ronald J. Adams 

2. Session B: Scene Content and Simulator Training Effectiveness 

Walter W. Johnson 

3. Session C: Low-Cost Training Alternatives: Part- and Full-Task Trainers 

David A. Lombardo 

4. Session D: Dynamic Response and Engineering Fidelity in Simulation 

Edward D. Cook 

5. Session E: Current Training: Where Are We? 

Greg J. McGowan 

6. Session F: Aero Modeling 

Ronald W. Du Val 
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1. SESSION A 


TRAINING: LIMITS, ALLOWANCES, AND THE FUTURE 

RICHARD J. ADAMS,* MODERATOR 


Panelists. Edward Boothe, FAA National Simulation Program Office; Martin Flax, Northrop Corporation; Edward 
Stark, Research Consultant; Curt Treichel, United Technologies, Inc. 


Principal Topics. Improved training and safety using simulators; regulatory limitations on testing helicopter emer- 
gencies; reduced training and cost; recommended aeronautical experience flight proficiency regulatory changes; certifi- 
cation credit for improved simulator training; and working-group proposed revisions to airman certification regulations. 


Historically, the qualification, approval, and use of 
helicopter simulators have been constrained by the state of 
the art of visual-system fidelity and phase lag or motion- 
system performance. The effects of these technological 
limitations on the low-speed performance and hover char- 
acteristics of simulators have been to curtail the use of 
simulators for airman certification purposes. The intent of 
this session was to develop a statement of user needs for 
simulators, to analyze the skills pilots need to do their 
jobs, and to examine the suitability of presently available 
simulators and motion and visual systems. 

Rather than dealing specifically with the principal 
topics originally suggested for Session A, the panelists 
encouraged a wide-ranging, open discussion as a means of 
getting ideas presented and discussed and eliciting com- 
ments and criticisms. 

Warren Robbins (FAA Flight Standards) reviewed 
the proposed Title 14, Code of Federal Regulations, 

Part 142. The proposed Part 142 provides for a much 
more extensive use of helicopter simulators and of other, 
various-level training devices. Given this new regulatory 
sanction, it will remain for each affected school to 
develop an appropriate training program and associated 
syllabi. Once the programs and syllabi are approved, the 
schools will be free to market their products. Ed Boothe 
also pointed out that the new Part 142 certified training 
school would afford much greater flexibility in the ways 
in which helicopter simulators could be used. 


‘Advanced Aviation Concepts. 


The Session A group agreed upon four recommenda- 
tions. First, it agreed to support Ed Boothe in his efforts 
to bring out the advisory circular. This support would 
extend beyond the workshop to provide support for the 
philosophy underlying the circular. 

Second, it was agreed to support changes in the pro- 
posed MPRM for Part 142 when that document is made 
available for public comment. Warren Robbins is making 
every effort to produce a good and useful document, and 
the timely submission of panel comments will facilitate 
his work. 

The third recommendation had to do with exemp- 
tions, in particular with supporting and encouraging Greg 
McGowan in his pursuit of additional exemptions that 
would enable the further utilization of the FlightSafety 
simulators. Special emphasis was placed on his attempts 
to gain approval of the simulator for use in granting add- 
on ratings. Some of those present agreed to work with 
Greg in producing another letter requesting this latter 
exemption. Greg said that FlightSafety is presently seek- 
ing an addendum or change to 4609 that would allow 
them to do some of the things discussed at the workshop. 
Other efforts along this line are more or less on hold, 
pending the outcome of this request. 

The near-term plan in this regard is to request as 
many exemptions as practical while supporting longer- 
term objectives. The issue of treating the simulator as a 
training tool, just as the aircraft is, generated extensive 
discussion. It is recognized that both these training 
devices have limitations, but it is important to recognize 
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that bath have real and useful capabilities that should be 
appropriately exploited. 

It would be desirable if the regulations would permit 
rating approval, testing, and licensing approval for simula- 
tors, if that is what the student has access to, or for the 
aircraft, if dial is available to him. If he fails in the simula- 
tor, however, he fails just as certainly and to the sajne 
extent as if he had failed in the aircraft. 

Fourth, it was recommended that the simulator be 
used as a crew training and evaluation tool. Because 


industry is moving more and more toward the use of 
simulators and because interpersonal skills and resource 
management are key safety factors, it was agreed that 
these skills could best be evaluated in a simulator. 

Although Session A discussions did touch on the 
issue of levels of sophistication that simulators would 
have to possess before flight-hour credits could be given, 
this matter was not considered in detail and was left for 
future meetings. 
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2. SESSION B 


SCENE CONTENT AND SIMULATOR TRAINING EFFECTIVENESS 


WALTER W. JOHNSON,* MODERATOR 


Panelists. Robert Hennessy, Monterey Technology, Inc.; Robert J. Randle, NASA Ames Research Center. 


Principal Topics. Scene display technology; scene image content; simulator utilization; and compliance evaluation. 


The single most important feature of modern flight- 
training simulators is their visual systems, but relatively 
little work has been done to determine precisely what 
scene content best supports the training functions they 
perform. 

The emphasis has been on how well scenes are drawn 
(resolution) and on how fast they are drawn (update rate). 
The principal purpose of Session B was to promote a dis- 
cussion of how the physical scene presented by the 
simulator — for example, terrain, clouds, and objects — can 
influence the effectiveness of simulator training. 

The panel addressed two main issues related to scene 
display technology in helicopter simulators: the impor- 
tant ways in which this technology can affect depth per- 
ception, and minimal field-of-view (FOV) requirements in 
simulators. 

For performing low-level, close-in missions, as heli- 
copters are often required to do, appropriate depth cues 
are viewed as being of major importance. In this regard, 
both collimated displays and the absence of binocular dis- 
plays were discussed. Collimated optics, which cause all 
displayed objects to appear at a great distance, generate a 
compelling feeling of depth in the displayed scene. How- 
ever, this is optically correct only for simulating objects 
that are far way from the observer, and thereby conflicts 
with scene content information — for example, perspective 
and absolute size — in which the objects are shown at 
shorter ranges. Consequently, there were recommenda- 
tions by panel members that (1) a thorough analysis of 
image collimation be conducted to determine how it 
affects or distorts the appropriate optics for near-objects, 
with particular attention to different eye positions 


* 
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(pilot/co-pilot) or observer head movements; and 
(2) human performance studies be undertaken to evaluate 
the importance of these depth cues and their accompany- 
ing distortions. 

The need for good binocular cues was considered, 
with many panel members saying these cues are essential 
in low-level, close-in tasks. It was pointed out that we are 
capable of testing the importance of binocular cues in 
many head-slaved systems, but have not yet done so. 
Because providing binocular cues will, of necessity, 
require head-mounted displays, a significant cost will be 
incurred. Nonetheless, some of the researchers involved in 
this work consider the provision of binocular cues a 
potentially critical factor in close-in work capability for 
helicopters. 

Field-of-view (FOV) requirements are considered to 
be an essential issue by industry and research workers. 
Because displays are the major cost items in simulation 
systems, industry needs to know what the requirements 
are. The panel did not find a consensus on this matter, but 
several related points were brought up during the 
discussions. 

It was agreed, for example, that FOV requirements 
are largely maneuver-dependent, and that the horizontal 
FOV must exceed 140°, although the need for FOVs 
greater than 180° was questioned. It was also noted that 
although many pilots want a vertical FOV, such a capa- 
bility is often unavailable in the actual aircraft. Moreover, 
vertical field of view is most often a function of cockpit 
design, but is also dependent on the rolling and pitching 
that are often encountered during maneuvers. For exam- 
ple, during decelerating landings, the helicopter often 
pitches up, thus eliminating any forward views looking 
downward, even though the cockpit design allows such a 
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view in other orientations. Consequently, the important 
thing is to make the FOV in the helicopter simulator 
appropriate to the helicopter and to the maneuvers being 
simulated. 

The panel discussion of scene image content was 
concerned principally with the issues of realism and gen- 
eralization. The FAA representative wants the scene to be 
as realistic as possible, the reason being to make the simu- 
lator capable of doing final check rides, especially for 
approaches into urban vertiports, and of allowing the pilot 
to fly as well in the simulator as he would in the heli- 
copter. In this regard, there were discussions about scenes 
depicting specific areas (e.g., a vertiport in a given city) or 
if the effort should be, instead, to present general charac- 
teristics from a range of possibilities. 

Unfortunately, the desire for maximum realism is in 
conflict with training uses of the simulator, in which the 
ability to use both generic visual cues (horizon ratios, tex- 
ture density, known size scaling) and special cue training 
paradigms was considered of utmost importance. Again, 
the panel concluded that the lack of essential research into 
some of these topics made it difficult to establish a visual 
data base that would at once be optimum for training and 
for efficient pilot certification testing. 

In the panel’s discussion of the utilization of simula- 
tors it was noted that training (initial, transitional, and 
recurrent) and certification applications of helicopter 
simulators may well require significant differences in the 
visual scenes used. For example, training effectiveness is 


often improved by selective manipulations of the visual 
scene, whereas certification testing requires highly stan- 
dardized formats. 

The panel members acknowledged that certification 
of compliance is a difficult issue. Some thought that the 
introduction of compliance requirements for the visual 
scene data-base design would result in prohibitive cost 
increases. As a result, it was proposed that consulting 
groups of experts should be the recommended approach. 

It was also proposed that methods should be devel- 
oped for evaluating compliance; expert opinion is an 
example of this approach, but performance-based criteria 
were also suggested. The point was made that it is diffi- 
cult to have principled compliance criteria without mea- 
surements of in-flight pilot performance as a basic 
reference. 

FAA representatives involved in TERPS develop- 
ment want simulators to permit performance as good as 
that that can be achieved in flight, thereby ensuring that 
TERPS criteria can be met during certification flights. 
However, others cautioned that this performance must not 
be achieved by making the simulation unrealistic. 

In general, the panel members agreed that the tech* 
nology exists to provide the visual scene content that is 
required in simulations, but that we do not yet know what 
we should put in the visual data bases. Similarly, the dis- 
play technology required for close-in helicopter missions 
has not been explored. Both of these deficiencies must be 
addressed in a more direct manner. 
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3. SESSION C 


LOW-COST TRAINING ALTERNATIVES; PART- AND FULL-TASK 

TRAINERS 


DAVID A. LOMBARDO/ MODERATOR 


Panelists. Graham Beasley, Silicon Graphics; Jack Dohme, U.S. Army Research Institute; Steve Hampton, Embry- 
Riddle Aeronautical University; Alfred Lee, Integrated Systems Engineering. 


Principal Topics. Personal computers and training software; computer-based flight-training devices; building the 
modular simulator; and designing for training device effectiveness. 


The purpose of Session C was to review the back- 
ground of and current research efforts in the general area 
of low-cost, computer-based simulation alternatives, as 
well as to provide recommendations for directing future, 
related work. Low-cost simulation alternatives are defined 
to include computer-based flight simulation, and both 
generic and type-specific non-motion flight-training 
devices. These alternatives have been used by many 
fixed-wing operators with great success, but have been 
otherwise ignored because they do not meet FAA guide- 
lines for flight-training devices and simulators. Nonthe- 
less, technological advances in the microprocessor indus- 
try ensure that the training capabilities of these and similar 
devices will be moved forward in directions and ways that 
are as of now unimaginable. 

Session C panel members from Silicon Graphics, the 
U.S. Army Research Institute at Fort Rucker, the Embry- 
Riddle Aeronautical University, Integrated Systems Engi- 
neering, and Bowling Green University presented a series 
of wide-ranging papers. There were discussions, among 
others, of what constitutes a “low-cost” alternative, and 
what can be expected of them in terms of capability. 

Research concluded earlier this year — in which 
computer-based training and computer-based flight simu- 
lation and their applications to teaching instrument proce- 
dures and, primarily, navigation procedures (e.g., VOR, 
ADF) — was reviewed. Overviews of low-cost training 
devices and a summary of a project in which computer- 


based training was specifically applied to attitude-instru- 
ment flying were presented. In the latter, emphasis was on 
ab initio students and the extent to which they could be 
effectively trained in attitude instrument flying through 
use of a computer-based training program. 

Regarding personal computers and training soft- 
ware, there was a consensus that they are worthy of addi- 
tional support. Panel members viewed them as a develop- 
ing technology, a way of the future. Virtual reality was 
discussed, and what is viewed as its major implications 
for simulation was summarized. 

The panel’s discussion of computer-based flight- 
training devices was never developed owing to time con- 
straints. In discussions of the modular simulator, there 
was general agreement about a generic type of data base 
and that a reasonable amount of vertical information 
would have to be presented for effective helicopter train- 
ing. The discussion here dealt principally with the data 
base itself, with only limited consideration given to such 
items as the number of channels required and the ones 
used the most. 

The fourth topic for the panel’s consideration was 
designing for training device effectiveness. In brief, the 
discussion of this topic reduced itself to a question: Are 
the data available that would permit a reliable prediction 
of the training effectiveness of a given simulator without 
the need of evaluating the simulator? The panel concluded 
that the answer is no — the data do not exist. 


*Bowling Green State University. 
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4. SESSION D 


DYNAMIC RESPONSE AND ENGINEERING FIDELITY IN SIMULATION 


EDWARD D. COOK," MODERATOR 


Panelists. Richard Bray, NASA Ames Research Center; Roger Hoh, HOH Aeronautics, Inc.; A1 Sodergren, FAA. 


Principal Topics. Maximum tolerable transport delay; handling-qualities parameters; other matching parameters; 
motion parameters; and visual system parameters. 


Simulator responses to control inputs must duplicate, 
within specified tolerances, the responses that the same 
inputs would effect in the actual aircraft. It is these 
tolerances that constitute the main subject matter in the 
development of simulator standards. The responses that 
are usually measured are of three general kinds: (1) air- 
craft responses of the kind frequently used to measure 
handling qualities; (2) limiting transport delays; and 
(3) correct motion responses. These responses have long 
been used to ensure adequate simulator fidelity so that 
pilot skills learned in the simulator transfer to aircraft. The 
application of these methods to helicopters re-opens the 
issues discussed in Session D. 

Regarding transport delay, the panelists first dealt 
with the question of what constitutes an acceptable delay. 
The consensus seems to be for a transport delay of about 
100 msec. There are commonly used methods for reduc- 
ing the effects of transport delay by adding lead to the 
system. Which brings up the question of how to check the 
efficiency of such methods. Should it be done in the fre- 
quency domain? For example, should it be done using a 
sine wave or with a step input? The problem is that delay 
can be compensated for with a lead circuit in the fre- 
quency domain only so long as there is a fairly smooth 
and continuous input. 

Given a sudden step, however, there will still be a 
temporary delay in the transport delay. One panel member 
recommended that the phase delay parameters used in 
handling-qualities analyses are potential metrics for 
determining whether the simulator properly represents its 
stability characteristics to the pilot as a whole. And this 
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leads to the question of defining appropriate handling- 
qualities parameters and selecting the correct ones to use. 

The panel agreed that the frequency-response data of 
the total end-to-end system are probably as good a crite- 
rion as any for determining whether the system is working 
together as a unit and whether it represents itself to the 
pilot as the real system it simulates. 

Do the Cooper-Harper ratings serve as a good basis 
for comparisons? In terms of validating a simulation, is 
the practice of having a pilot rate both the simulation and 
the flight vehicle a reasonable one? The panel decided 
that the answer is probably no. Comparisons of that kind 
are seen as being too time-consuming and too costly. 

Concerning which motion parameters should be 
specified, the panelists agreed that the bandwidth of the 
motion hardware, which is a limiting factor, has to be 
increased. If it is not possible to do a good, all-around job, 
the yaw axis, vibration, and on-ground contact were 
selected as being the most important motion cues and the 
ones that should be emphasized. As for latency in the 
visual system, it can, to a great extent, be corrected by 
prediction techniques. The sudden step will still cause 
delay, but when the motion is continuous, the delay can be 
led and tracked, 

It was agreed that the actual latency that must be cat- 
egorized should be task-dependent and driven by the stip- 
ulated level of certification. That is, there should be no 
one generalized number, And a final comment — there 
were suggestions that a 30° field of view downward 
through the chin window should be provided. 
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The panel discussion also touched on the problem of 
the relative difficulty of flight in the simulator and in the 
aircraft. For example, hovering is probably more difficult 
in the simulator than in the aircraft, and there was some 


talk about providing subtle augmentation in the simulation 
as a means of making the workloads in the simulator and 
aircraft more equivalent. 
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5, SESSION E 

CURRENT TRAINING: WHERE ARE WE? 

GREG J. MCGOWAN,* MODERATOR 


Panelists. Kenneth Cross, Anacapa Sciences; Gerald Golden, Petroleum Helicopters, Inc.; Douglas Schwartz, 
FlightSafety International. 


Principal Topics. Areas of simulation improvement; simulation utilization; and economics and accessibility. 


The use of commercial helicopter simulators for 
training and checking is controlled by FAA regulations 
(FARs). In some cases, exemptions to the FARs are 
granted for the use of an approved simulator; in others, 
the FARs themselves permit use of the simulator for spec- 
ified training and checking procedures. However, many 
procedures — especially emergency procedures — that are 
routinely practiced in simulators, are not required by the 
FARs. As a result, the simulator’s capacity for training 
that goes beyond the scope of the FARs is being 
underutilized. 

The Session E panel convened to discuss, in general, 
the three principal topics mentioned above — areas of 
simulation improvement, simulation utilization, and the 
economics and accessibility of helicopter simulators. In 
the event, however, the discussion centered on the third of 
those topics, the economics and availability issue, modi- 
fied, however, to couple the “helicopter simulator” term 
with “training devices.” 

Most of this panel’s discussion pertained to training 
devices and to what can be done, especially for devices 
that rank in capability (and thus in complexity and cost) 
below the approved simulator level, to enhance their 
availability and to make them more economical to use and 
maintain. 

The panel agreed that the benefits of helicopter simu- 
lation training can be made economically available to a 
larger segment of the helicopter-user community only 
through use of training devices that offer a range of train- 
ing capabilities. Two prerequisites to ensure that such an 
expansion in training-device availability occurs were 
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identified: (1) definition of the training objectives and 
tasks that these devices would or should address, and 
(2) the development of an advisory circular, or of an 
appendix to an existing advisory circular, that would set 
forth the criteria with which the various levels of training 
devices would have to comply. 

A third step, an outgrowth of item (2) above, identi- 
fies the need to establish the training and checking 
requirements that will be allowed for each level of train- 
ing device. That is, a determination must be made about 
how these devices will be certified and about the training 
uses for which they will be approved. 

These are essential considerations for the training- 
device manufacturers. If these devices are going to be 
made available to the operators, the manufacturers must 
have assurance that the devices they propose to produce 
are going to meet preestablished criteria of acceptability. 
If such criteria are not set forth, the risk of manufacturing 
the devices is too great to be entertained. 

In the panel’s discussion of the above, another ques- 
tion surfaced: Would the envisioned proliferation of 
lower-level training devices act to stifle the development 
of the more elaborate and technologically superior 
simulators — the Level C and D simulators? The panel’s 
answer was no — the widespread use of limited-capability 
but effective training devices will not cut into the market 
for the highly capable machines. On the contrary, the 
panel finds it likely that the lower-level training devices, 
by introducing operators to the possibilities of simulation 
training in general, will act as a market stimulant for the 
more advanced (and expensive) simulators. As more and 
more operators use the lower-level devices, their interest 
in the higher-level ones will be heightened and they will. 
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perhaps, come to constitute a new market segment for the 
full-fledged simulator. 

Although not an agenda item for this session, the 
panel discussed the issue of transfer of training. As a 
result, it recommended a thoroughgoing review of all 
studies pertaining to the transfer of training from simula- 
tor (or training device) to the actual flight vehicle. That is, 
does a skill mastered on the training device transfer posi- 
tively and directly to operation of the aircraft? All sources 
of such information should be exploited — government, 
military, commercial, domestic or foreign. 

If the review discloses that the information on trans- 
fer of training is inadequate for purposes of making reli- 
able conclusions, the panel recommended that an appro- 
priately designed study, one of adequate scope to ensure 
comprehensive data production, be conducted. It is a 


given that transfer-of-training studies are difficult and 
expensive. Consequently, it is the panel’s suggestion that 
such an effort be undertaken with the full cooperation of 
the government, industry, operators, and users. 

In conjunction with any transfer-of-training study, 
there is a need for a well-defined helicopter job-task 
analysis. A previously conducted job-task analysis identi- 
fied 56 jobs that are now being done with helicopters. The 
panel’s recommendation is that these helicopter jobs be 
analyzed and broken down into their component tasks. 
Then, given the results of a comprehensive transfer-of- 
training study, the most effective training devices or 
simulators can be matched with the training needs at hand 
to produce the most effective and economical training. 
Task-designated priorities would ensure that tasks having 
the most direct bearing on safety would be addressed first. 


6. SESSION F 


AERO MODELING 

RONALD DU VAL* MODERATOR 


Panelists. Frank Cardullo, State University of New York; R. Thomas Galloway, Naval Systems Training Center; 
Robert Toller, Quintron; Gary Hill, NASA Ames Research Center. 

Principal Topics. Physically based simulation models; validation of physically based models; and achieving a 
higher level of physical modeling simulation. 


Trainer manufacturers typically relay heavily on 
empirical models as a means of reducing computation 
time and maximizing tunability. Unfortunately, these 
models may provide poor fidelity away from the test 
points, and this is particularly true of rotorcraft simulators, 
in which empiricism may mask additional degrees of 
freedom as well as severe nonlinearities. This panel’s 
purpose was to consider the need for an increased level of 
physically based modeling in rotorcraft simulators. 

The panel’s discussion centered on the trade-offs 
between physical and functional modeling for training 
simulations. It became clear during the discussion that 
terms had to be better defined, and from that evolved a 
better understanding of what is meant by an acceptable 
form of functional simulation: one traceable to first prin- 
ciples through a physical simulation. 

For example, there are instances in which a rotor-map 
model may be an appropriate simulation model. And as 
long as the rotor-map model is traceable to a blade- 
element model from which it was derived, the 
functionality can be traced. That is to say, if at first a 
physical model of the system is created, and if the 
necessary approximations and reductions are made to 
bring it down to an appropriate level for the task to be 
undertaken, it should then be possible to track it back to 
the higher-level engineering model; in that way, control 
can be maintained over the procedures used to provide the 
modeling. 

The other level of traceability is through experimental 
data; for example, modeling an airfoil in terms of lift and 


* Advanced Rotorcraft Technology. 


drag data that are traceable to a wind-tunnel test. The 
point is that the level of functionality or analytical model- 
ing present at any point in the simulation has to be depen- 
dent on the tasks performed on the simulator and on the 
level of certification that the simulation is intended to 
support. But for comparative purposes, it should be pos- 
sible to trace any functional representation to a higher 
physical level so that the assumptions involved and the 
conditions under which the functionality is valid can be 
known. 

The trade-offs concerning rotor-map and blade- 
element models were considered in this session, and it 
was concluded that the magnitude of the computational 
task associated with the blade-element model is no longer 
a significant limitation in its application to training 
simulation. Although in the past the computation costs of 
the blade-element model were prohibitive, fast parallel- 
processing computers are available and are up to the 
computational tasks involved. As a result, decisions 
concerning the use of one or the other of these models 
should no longer be based on computation costs. 

Choosing between rotor-map and blade-element 
models means considering model tunability. The rotor- 
map model is easier to tune as a means of complying with 
acceptance criteria, but whether that is desirable or not has 
not been resolved. If model performance is force-fitted to 
comply with acceptance criteria, it is no longer a physi- 
cally based model, and its validity between test points is 
unknown. Tuning the blade-element model, on the other 
hand, requires validation from physically meaningful 
parameters or from model structure changes, which is a 
much more costly process. 
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Interactional aerodynamic and inflow models were 
viewed as comprising an important problem area, but one 
that is commonly neglected in training simulations. The 
empirical models that are used to cover these problems 
are often inadequate. It was noted that there is a require- 
ment for shipboard landing simulations that can properly 
account for the aerodynamic interactions of the rotor wake 
during approaches to rolling ship decks and for the inter- 
ference of the ship superstructure with the aircraft. Even 
engineering-level simulations lack adequate modeling to / 
properly assess these issues. The solutions to some of 
these problems await technological developments. 

Other issues involve solution and integration tech- 
niques. Although not usually set out in the acceptance test 
criteria, these factors nonetheless significantly affect 
simulator performance. The questions here are whether 
degrees of freedom are to be solved simultaneously or 
sequentially, how large an integration step size to use, and 
what kind of integration algorithms to use. The alterna- 
tives are many, perhaps to such an extent that they con- 
tribute to the problem — there are so many different 
approaches that can be pursued. 

The panel's discussions emphasized the advantage of 
subsystem-by-subsystem validation over complete end-to- 
end validation of the entire system. With the former, what 
is required and how it is required can be stated more 
specifically. Instead of looking at the aircraft response to 
stick movement, for example, one looks at the way the 
rotor responds. Isolating the various components of the 
simulation model and validating them individually, 
improves the flexibility with which simulation models can 
be interchanged in future machines. As a result, one 
would not have to start anew with each simulation. More- 
over, there would be greater confidence that the model 
was correct off test points. 


The way models are validated also affects the physi- 
cal model. A simulation may begin with a lot of physical 
content but then have a whole structure of tuning coeffi- 
cients superimposed on it when it comes to meeting the 
acceptance test criteria. At present, procurement spec- 
ifications do not prohibit the manufacturer from using this 
means of passing the acceptance test. So perhaps consid- 
eration should be given to specifying which parameters 
can be tuned, thus making certain that it is done in a phys- 
ically meaningful manner. 

How can contractors be required to use a higher level 
of engineering analysis and fidelity in their training mod- 
els? One way would be to specify that each subsystem be 
validated separately and to specify the acceptance tests in 
terms of frequency-domain criteria. For example, specify- 
ing the frequency response of the rotor with respect to 
motions of the hub would mandate a blade-element 
model; accurate frequency-response data for the rotor 
could not be achieved with a rotor-map model. 

A final and valid question that came out in the 
panel’s discussion: Why create fine physically based 
models when the control system completely overwhelms 
the physical aspects of the system? It is true that the pilot 
cannot appreciate what is going on because of the heavy 
suppression of the control system. This leads to another 
question: Should a simulation be validated only for the 
nominal flight condition — control system on in the middle 
of the envelope with mild maneuvering — or should 
training systems be validated to properly model extreme 
conditions? If the latter is the goal, that is, if control- 
system failures, edge-of-the-envelope maneuvering, and 
other aggressive maneuvers are to be modeled, the 
mathematical basis for the simulation has to be far more 
sophisticated. 
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